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Method, computer program and system of visualizing image data relating to a medical 
examination of a subject EPO - DG 1 

2B.rn.2W 



£4j 

The present invention relates to a method of visualizing image data relating to 

i 

medical examination of a subject, comprising the step of automatically selecting one or more 
appropriate protocols from a set of predefined protocols defining visualizing techniques to be 

applied to the image data. 

Medical image data of a subject, notably a (human) patient, can be acquired by 
various medical examination techniques, such as X-ray techniques, Magnetic Resonance- 
techniques, Computed Tomography techniques or ultrasound techniques. After acquisition 
the image data need to be presented in an appropriate way to medically skilled personnel for 
review. 



A method of the type as described above is known from US 5,986,662. 
According to the known method an appropriate display protocol is selected based on attribute 
data that accompany the image data. Attribute data may comprise the name of the 

15 examination technique performed that needs to be filled in by the user or may be filled in 
automatically when scanning the patient. Other attribute data comprise the number and 
resolution of the images and the modality used. 

The known method has the disadvantage mat the attribute data to be filled in 
are often not filled in completely. Moreover there is no accepted standard to describe each 

20 medical examination. The other attribute data mentioned are usually not sufficient to decide 
on the nature of the examination. In case the nature of the examination is unsure default 
display techniques may be applied to the image data resulting in a sub-optimal presentation. 
In order to enhance the presentation the user will have to perform a manual selection of the 
appropriate protocol, which can be a real burden, especially for the less experienced user. 

25 



It is an object of the method according to the invention to provide a method of 
the type as described above that overcomes this disadvantage. 
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The method according to the invention is therefore characterized in that the 
method further comprises the steps of: b) analyzing the image data; c) deciding on the part of 
the subject's anatomy represented by the image data; and/or d) deciding on the purpose of the 
medical examination performed on the subject; and e) selecting one or more of the 
5 appropriate protocols in dependence of the anatomy part present and/or the purpose of the 
medical examination performed. 

By analyzing the content of the image data an appropriate protocol can always 
be selected without intervention from the user. The method according to the invention thus 
allows for a more automated and flexible workflow in the medical profession. 
10 In a first preferred embodiment step e) comprises the step of: el) selecting one 

or more appropriate protocols from a set of predefined protocols, a number of said predefined 
protocols defining processing techniques to be applied to the image data. Advantageously 
herein also suitable processing techniques are automatically applied to the image data 
ensuring an optimal presentation of the image data to the user and a further automated 
15 workflow. 

According to a second preferred embodiment of the method according to the 
invention step e) comprises the step of: e2) automatically selecting one or more appropriate 
protocols from a set of predefined protocols, a number said predefined protocols defining 
techniques for Computer Aided Diagnosis (CAD) to be applied to the image data When 

20 starting the necessary CAD techniques automatically the workflowis even .further automated. 
Since CAD techniques are usually time consuming this shortening of the start-up time allows 
for a quicker review of the image data. 

According to a third preferred embodiment the method according to the 
invention step e) comprises the step of: e3) selecting one or more appropriate protocols from 

25 a set of predefined protocols, a number of said predefined protocols defining anatomy 
dedicated techniques to be applied to the image data. The workflow is now successfully 
automated even further by automatic selection of so called "dedicated tools" that are 
developed for visualizing specific parts of the human anatomy and are known per se in the 
relevant field. 

30 Preferably step e) of the method according to the invention further comprises 

the step of: e4) automatically selecting one or more appropriate protocols from a set of 
predefined protocols, a number said predefined protocols defining display techniques to be 
applied to the image data. The choice of a suitable display technique is now automatically 
made and the image data are presented in a correct view to the user. 
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In another preferred embodiment step b) comprises the step of comparing the 
image data to reference data. By using reference images the analysis of the image content can 
be conducted in a reliable and quick manner. 

In further preferred embodiment step b) comprises the step of subdividing the 
image data in coherent parts on the basis of expert knowledge. Introducing expert knowledge 
in the analysis advantageously results in an even more reliable outcome. 

According to yet a further preferred embodiment step b) comprises the step of 
extracting salient structures present in the image data. Extracting and defining salient 
structures of the human anatomy, like bones, contrasted vessel, air filled colon, etc, is of great 
value in me analyzing step. It is also an important preparation for step c) defining the 
anatomy present in the image data and for step d) deciding on the purpose of the medical 
examination performed on the subject. 

The invention also refers to a computer program to carry out the method 

according to the invention. 

The invention also concerns a system to carry out the method according to the 
invention, comprising: means for automatically selecting one or more appropriate protocols 
from a set of predefined protocols defining visualizing techniques to be applied to the image 
data. The system is characterized in that it further comprises: b) means for analyzing the 
image data; c) means for deciding on the part of the subject's anatomy represented by the 
image data; and/or d) means for deciding on the purpose of the medical examination 
performed on the subject; and e) means for selecting the appropriate protocol in dependence 
of the anatomy part present and/or the purpose of the medical examination performed 



The invention will be further explained by means of the attached drawings, in 

which: 

Fig. 1 shows a diagram illustrating the main steps of the method according to 
the invention; and 

Fig. 2 schematically shows the main components of a system to carry out the 
method according to the invention. 



In general the method according to the invention refers to a method of 
visualizing medical image data relating to a subject Fig. 1 shows a diagram illustrating the 
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main steps 1 0, 20 and 30 of the method according to the invention that will now be discussed 
in detail. 

Step 10: Analyzing the image data: 

In this step the content of the image data is analyzed. Many analyzing 
techniques are known per se in the art and can be used in step 10. In the following a number 
of them will be discussed that are believed to be especially useful. 

As a first example of an analyzing technique the globed or local gray value 
histograms in the image can be determined. Another example would be to asses the 
distribution of features in the image data. Various features can be looked at, such as 
transitions, edge strengths, texture, noise structure etc. 

As another example of an analyzing technique salient structures present in the 
image data can be extracted. Salient structures typically comprise anatomic landmarks, such 
as sharp edges and strong transitions. Extraction of main shapes and their sizes may also be 
useful. 

The analysis of the mage data may further comprise the step of comparing the 
image data to reference data comprised in a database. Suitable reference data typically 
comprise image data of which the content is known. Furthermore or as an alternative the 
analysis of the mage data may comprise the step of subdividing the image data in coherent 
parts on the basis of expert knowledge. Expert knowledge can for instance be introduced by 
using predetermined threshold data values; e.g. based on Hounsfield values, e.g. for CT 
exams, relating to transitions in the image data, such as the transition from air to soft tissue. 

Step 20: Deciding on anatomy present and/or examination purpose: 

Based on the outcome of the analysis in this step a decision should be made 
regarding the content of the image data, either regarding the anatomy that is most likely 
incorporated in the image data or the purpose of the examination performed, or both. 

In the field of computer science decisions are generally taken by defining a 
decision tree and walking through it. In such a decision tree a number of relevant questions 
are defined. Different combinations of positively answered question will lead to different 
outcomes. As will be apparent the number of possible outcomes may in one embodiment 
correspond to the number of anatomy parts of the human body, which is too large to cover in 
detail in this application. As an illustration therefore a limited number of combinations 
leading to definition of specific anatomy parts are discussed now. 
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IF image content = vertebras 

AND image content = two large volumes of air 

THEN anatomy = thorax 

IF image content = vertebras 

AND image content - tube like structure with air 

THEN anatomy = abdomen AND examination purpose = colon 

IF image content = (half)round shape 

AND image content = small tube like structures 

THEN anatomy - head AND examination purpose = vascular study 

In case additional data are available, for instance in the form of attributes, such 
decision making step, as illustrated in the following step. 

IF examination technique = MR 

AND number of images = large 

AND time series = short 

AND image content = two concentric circles 

AND size of two concentric circles = constant 

THEN anatomy = heart AND exam = perfusion 

IF examination technique = MR 
AND number of images = large 
AND time series = short 
AND image content = two concentric circles 
AND size of two concentric circles = variable 
THEN anatomy = heart AND examination purpose = functional analysis 

) 

Step 30: selecting the appropriate protocol: 

Based on the anatomy decided to be present or on the decided purpose of the 
examination performed or onthe combination thereof, in this step the appropriate protocol 
should be selected. A protocol may comprise display techniques or processing techniques or 



PHNL020355EPP 



i 



6 25.04.2002 
CAD techniques or dedicated tools, or any combination thereof. Typically the protocols are 
present as software applications that can be applied to the image data in order to provide the 
user automatically with a correct view of the image data. The protocols can preferably be 
configured by the user so as to incorporate preferred tools, viewing and/or processing 
5 techniques for each type of examination. A number of protocols are discussed in more detail 
below. 

Type of protocol: 

The set of predefined protocols will preferably include a number of display 
protocols defining display techniques to be applied to the image data. Some examples of 
1 0 useful display techniques comprise: reformats (slice o f data), Maximum Intensity Projection 
(MIP), volume rendering techniques, surface rendering techniques. All of these display 
techniques are known per se in the relevant field. 

The set of predefined protocols will preferably include a number of processing 
protocols defining processing techniques to be applied to the image data. Some examples of 
15 useful display techniques are mentioned below, all of which are known per se to a person 
skilled in the art 

- Background processing, e.g. defining background pixels and assigning 
suitable values thereto; 

- Path tracking, e.g. for defining the center of or the centerline through a part 
of the subjects anatomy, such as a vessel or a colon; 

- Segmentation of (part of) the anatomy, such as bone, optionally followed by 
removal of one or more segments, f.i. bone removal; 

- Measurements of anatomy parts, such as measuring the diameter of a vessel, 
the size of a polyp, or the volume of the left ventricle. 

The set of predefined protocols may preferably include a number of CAD 
protocols defining techniques for Computer Aided Diagnosis (CAD) to be applied to the 
image data. 

Some examples known per se relate to the search for abnormalities, such as 
aneurysms, colorectal polyps, tumors, lung nodules, abnormalities in a mammography. 

The set of predefined protocols may preferably further include a number of 
dedicated protocols defining anatomy dedicated techniques, also known in the art as 
"dedicated tools", to be applied to the image data. Some examples of dedicated tools are: 
virtual endoscopy tool (Endo-3d tool), heart inspection tool (Cardiac analysis tool), vessel 
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inspection tool (vessel analysis tool), lung nodule detection tool and lung nodule evaluation 
tool, etc. 

Selection of a protocol: 

The selection of a protocol can be realized by a simplified decision tree, as 
5 illustrated by the following examples. 

IF anatomy = thorax 

THEN display protocol = Digital Reconstructed Radiograph (DRR) 
AND CAD protocol = lung nodule detection tool 

10 

IF anatomy — head 

THEN display protocol = three orthogonal views 
AND display protocol = reformats 

j 5 IF examination purpose = colon 

THEN dedicated protocol = Virtual Endoscopy (VE) 

AND processing protocol = path tracking 

AND CAD protocol = polyp detection 



20 



IF anatomy = head AND examination purpose - vessels 
THEN display protocol - Maximum Intensity Projection (MIP) 
AND processing protocol = path tracking 
AND processing protocol = bone removal 

25 IF anatomy = heart 

THEN display protocol = reformats 

For illustration purposes two examples of the method according to the invention are given 
below. 

30 Example A: 

A CT data volume is loaded (this information is assumed to be present in the 

image attributes). 

In the analyzing step bony structures like spine and pelvis can easilybe 
isolated by means of gray value thresholding. After use of a rough structure recognizer me 
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decision can be taken that a spine and pelvis are present in the image data (at least partially). 
Based on me available infomiation the decision can be taken that the anatomy present is the 
abdomen, and only CT abdominal related protocols are selected. 

a) If an elongated, air-filled structure of a certain diameter is decided to be 

5 present, it can be decided that the purpose of the examination was colon and the patient was 
prepared for virtual colonoscopy. An appropriate protocol to be selected could comprise a 
dedicated tool, in this case an endo tool, as well as a CAD tool, in this case automatic polyp 
detection, as well as a processing tool, such as path tracking. 

b) If blood Vessels filled with contrast medium are decided to be present, an 

10 appropriate protocol to be selected could co mprise a dedicated tool, in this cas e a vessel 

analysis tool, as well as a processing tool, such as path tracking. 

c) If a contrasted aorta is decided to be present, an appropriate protocol to be 
selected could comprise a processing protocol for measuring the aorta automatically, and a 
dedicated tool can be started, for instance a CAD protocol in search of an aneurysm. 

15 Example B: 

An X-ray image is loaded (this information is assumed to be present in the 
image attributes). 

a) After use of a dedicated chest finder the decision can be taken that the 
anatomy present is the thorax. Only X-ray chest related protocols are selected. An 

20 appropriate protocol to be selected could comprise processing tools, such as enhancement 
filters, as well as a CAD tool, in this case an automatic lung nodule detector. 

b) The content analysis may point out that the anatomy present in the image 
data is the knee and the purpose of the examination is an orthopedic study. An appropriate 
protocol could comprise a processing protocol for taking measurements of the knee, for 

25 instance for knee prosthesis pl anning 

Now the method of the invention is explained a skilled person will be able to 
translate the steps of the method into a computer program to carry out the method. 

Fig. 2 schematically shows the main components of a system 1 that is arranged 
to carry out the steps of the method according to the invention. System 1 comprises an image 
30 data acquisition system 2. Many types of acquisition systems for acquiring medical data are 
available in the relevant field, among which are Computed Tomography systems, MR 
systems, X-ray systems, etc. All of these can form part of system 1 . 
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System 1 further comprises an image display system 6. Many types of suitable 
display systems ate available in the field, sueh as a personal computer, optionally functioning 

as a PACS station. 

According to the invention system 1 further comprises means 3, 4 and 5 
5 communicating with image data action system 2 and image display system 6. Meuns 3 
^ imaged for analyzing the image d*a. Menus 4 are arranged for deciding on the part of 
«he subject's anatomy that is represented by the image data and/or the purpose of the 
exm ^ 6oa performed on the subject Mean* 5 are arranged for selecting the appropnate 
protocol in dependence of the anatomy part present or the ptnpose of the examination 

10 performed. ^ ^ rf ^ fa to detaU a skilled person will be 

able to anunge means 3, 4 and 5, such that they are able to carry out me steps of the method 
All necessary means and techniques to accomplish this task are available per se to a person of 
ordinary skill in the art, either in software or in hardware. Means 3 require the most 

15 computational power and are therefore preferably coupled to a computer with mgh 
comnutational power, such as a workstation. 

1« is noted that means 3, 4 and 5 are drawn as separate units. It wifl however be 

clear to any person skflledinthe art that menns 3, 4 and 5 can also be combined info one or 
two units. As an alternative one or more of the units cen be incorporated into image date 
20 acquisition system 2 or into me image display system «. Means 4 and 5 could for example be 
incorporated into means 3 as well as into image display system 6. 

The invention is of course not limited to the described or shown embodiment, 
W generally extends to any embodiment, which falls within the scope of the appended 
claims as seen in light of the foregoing description and drawings. 
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CLAIMS: 2$. M. 2002 



! . Method of visualizing image data relating to medical examination of a subject, 

comprising the step of: 

a) automatically selecting one or more appropriate protocols from a set of 

predefined protocols defining visualizing techniques to be applied to the image data, 
5 characterized in that, the method further comprises the steps of: 

b) analyzing the image data (10); 

c) deciding on the part of the subject's anatomy represented by the image data 

(20); and/or 

d) deciding on the purpose of the medical examination performed on the 

10 subject (20); and 

e) selecting one or more of me appropriate protocols m dependence of the 

anatomy part present and/or the purpose of me examination performed (30). 

2 Method according to claim 1, wherein step e) comprises the step of: 
el) selecting one or more appropriate protocols from a set of predefined 

protocols, anumber of said predefined protocols defmmg processmg techniques to be appUed 
to the image data* 

3 Method according to claim 1 or 2, wherein step e) comprises the step of: 
20 ' e2)selecting one or more appropriate protocols from a set of predefined 

pr otocols,anumbersaidpredefined protocols definmg techniques for Computer Aided 
Diagnosis (CAD) to be applied to me image data. 

4 Method according to claim 1, 2 or 3, wherein step e) comprises the step of: 
25 e3) automatically selecting one or more appropriate protocols from a set of 

predefined P rotocols,anumber said predefined protocols defining anatomy dedicated 
techniques to be applied to the image data. 

5. Method according to claim 1, 2, 3 or 4, wherein step e) comprises the step of: 



15 
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e4) automatically selecting one or more appropriate protocols from a set of 
predefined protocols, a number said predefined protocols defining display techniques to be 
applied to the image data. 

6 - Method according to one or more of the preceding claims, wherein step b) 
comprises the step of comparing the image data to reference data 

7- Method according to one or more of the preceding claims, wherein step b) 
comprises the step of subdividing the image data in coherent parts on the basis of expert 
knowledge. 

8. Method according to one or more of the preceding claims, wherein step b) 
comprises the step of extracting salient structures present in the image data. 

9. Computer program to carry out the method according to one or more of the 
preceding claims. 



1 0. System to carry out the method according to one or more of the preceding 

claims 1 through 8, comprising: 

a) means for automatically selecting one or more appropriate protocols from a 
set of predefined protocols defining visualizing techniques to be applied to the image data, 
characterized in that the system further comprises: 

b) means (3) for analyzing the image data; 

c) means (4) for deciding on the part of the subject's anatomy represented by 
the image data; and/or 

d) means (4) for deciding on the purpose of the medical examination 
performed on the subject; and 

e) means (5) for selecting the appropriate protocol in dependence of the 
anatomy part present and/or the purpose of the medical examination performed. 
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The invention relates to a method of visualizing image data relating to medical 
examination of a subject. The method concerns automatic selection of appropriate protocols 
from a set of predefined protocols defining visualizing techniques to be applied to the unage 

5 ontheanalysisadecisionismaderegardmgthepartofthesubject's 

the image data and/or the purpose of the medical examination performed on the subject. 
Finally in the third step (30) one or more of the appropriate protocols are selected m 
dependence of the anatomy part present and/or the purpose of the examination performed 

The invention also refers to a computer program and a system to carry out the 

10 method according to the invention. 
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